One of the most popular uses of the Internet is to search for news reports. One survey conducted in 2004 indicated that 56% of the users listed news as one of the top five reasons for accessing the Internet. As the popularity of using the Internet to review news reports grows, the number of web-based news sources has proliferated. These news sources typically provide a portal or news home page that provides a summary of important news reports along with links to detail pages containing the news reports. Each news report may report one or more news events. For example, one news report may relate a bankruptcy filing of a company, and another news report may relate to a speech made by an elected official. A news home page may contain summaries of news reports at various levels of detail. For example, one or two important news reports may be prominently displayed on a news home page along with a title that serves as a link to a detail page of the news report and a two or three sentence summary of the news report. A news report of lesser importance may be displayed less prominently simply with a title that serves as a link to the detail page of the news report.
Because there are so many available news sources, it can be difficult for a user to identify news reports for important news events. A user could use a search engine service to search for recent news events, but without knowing the specifics of a news event, the user cannot effectively specify a search query for news reports reporting that news event. Moreover, search engine services may not be able to update their search mappings (described below) as quickly as needed to allow users to locate news reports as the news reports are posted by a news source. As a result, many users typically access a single news source (e.g., “MSNBC.com”). Unfortunately, a single news source may not have or prominently display all the news reports of interest to a user. As a result, a user may have no effective way to locate all the news reports of interest.
Using a search engine service, such as Google and Overture, a user may attempt to locate news reports published on display pages, such as web pages, that may be of interest to the user. After the user submits a search request (i.e., a query) that includes search terms, the search engine service identifies web pages that may be related to those search terms. To quickly identify related web pages, the search engine services may maintain a mapping of keywords to web pages. This mapping may be generated by “crawling” the web (i.e., the World Wide Web) to identify the keywords of each web page. To crawl the web, a search engine service may use a list of root web pages to identify all web pages that are accessible through those root web pages. The keywords of any particular web page can be identified using various well-known information retrieval techniques, such as identifying the words of a headline, the words supplied in the metadata of the web page, the words that are highlighted, and so on. The search engine service may generate a relevance score to indicate how relevant the information of the web page may be to the search request based on various metrics such as the term frequency and inverse document frequency metric (“tf*idf”). The search engine service may also generate an importance score to indicate the importance of the web page based on various metrics such as Google's PageRank metric. The search engine service then displays to the user links to those web pages in an order that is based on a ranking determined by their relevance and importance.
Two well-known techniques for determining the importance of web pages are PageRank and HITS (“Hyperlink-Induced Topic Search”). PageRank is based on the principle that web pages will have links to (i.e., “outgoing links”) important web pages. Thus, the importance of a web page is based on the number and importance of other web pages that link to that web page (i.e., “incoming links”). In a simple form, the links between web pages can be represented by matrix A, where Aij represents the number of outgoing links from web page i to web page j. The importance score wj for web page j can be represented by the following equation:wj=ΣiAijwi
This equation can be solved by iterative calculations based on the following equation:ATw=wwhere w is the vector of importance scores for the web pages and is the principal eigenvector of AT.
The HITS technique is additionally based on the principle that a web page that has many links to other important web pages may itself be important. Thus, HITS divides “importance” of web pages into two related attributes: “hub” and “authority.” “Hub” is measured by the “authority” score of the web pages that a web page links to, and “authority” is measured by the “hub” score of the web pages that link to the web page. In contrast to PageRank, which calculates the importance of web pages independently from the query, HITS calculates importance based on the web pages of the result and web pages that are related to the web pages of the result by following incoming and outgoing links. HITS submits a query to a search engine service and uses the web pages of the result as the initial set of web pages. HITS adds to the set those web pages that are the destinations of incoming links and those web pages that are the sources of outgoing links of the web pages of the result. HITS then calculates the authority and hub score of each web page using an iterative algorithm. The authority and hub scores can be represented by the following equations:
                    a        ⁡                  (          p          )                    =                        ∑                      q            →            p                                                          ⁢                              h            ⁡                          (              q              )                                ⁢                                          ⁢          and                      ⁢                              ⁢                                      h      ⁢              (        p        )              =                  ∑                  p          →          q                                              ⁢              a        ⁡                  (          q          )                    where α(p) represents the authority score for web page p and h (p) represents the hub score for web page p. HITS uses an adjacency matrix A to represent the links. The adjacency matrix is represented by the following equation:
      b    ij    =      {                            1                                      if            ⁢                                                  ⁢            page            ⁢                                                  ⁢            i            ⁢                                                  ⁢            has            ⁢                                                  ⁢            a            ⁢                                                  ⁢            link            ⁢                                                  ⁢            to            ⁢                                                  ⁢            page            ⁢                                                  ⁢            j                                                0                          otherwise                    
The vectors α and h correspond to the authority and hub scores, respectively, of all web pages in the set and can be represented by the following equations:α=ATh and h=Aα
Thus, α and h are eigenvectors of matrices ATA and AAT. HITS may also be modified to factor in the popularity of a web page as measured by the number of visits. Based on an analysis of click-through data, bij of the adjacency matrix can be increased whenever a user travels from web page i to web page j.